ThreadSanitizer: data race [@ nsZipArchive::BuildSynthetics] vs. [@ nsZipArchive::GetItem]
Categories
(Core :: Networking: JAR, defect, P2)
Tracking
()
People
(Reporter: jkratzer, Assigned: kershaw)
References
(Blocks 2 open bugs)
Details
(Keywords: testcase, Whiteboard: [bugmon:bisected,confirmed][necko-triaged])
Attachments
(4 files)
The attached crash information was detected while running CI tests with ThreadSanitizer on mozilla-central revision 20200730-0e4bc84faa30.
For detailed crash information, see attachment.
General information about TSan reports
Why fix races?
Data races are undefined behavior and can cause crashes as well as correctness issues. Compiler optimizations can cause racy code to have unpredictable and hard-to-reproduce behavior.
Rating
If you think this race can cause crashes or correctness issues, it would be great to rate the bug appropriately as P1/P2 and/or indicating this in the bug. This makes it a lot easier for us to assess the actual impact that these reports make and if they are helpful to you.
False Positives / Benign Races
Typically, races reported by TSan are not false positives [1], but it is possible that the race is benign. Even in this case it would be nice to come up with a fix if it is easily doable and does not regress performance. Every race that we cannot fix will have to remain on the suppression list and slows down the overall TSan performance. Also note that seemingly benign races can possibly be harmful (also depending on the compiler, optimizations and the architecture) [2][3].
[1] One major exception is the involvement of uninstrumented code from third-party libraries.
[2] http://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong
[3] How to miscompile programs with "benign" data races: https://www.usenix.org/legacy/events/hotpar11/tech/final_files/Boehm.pdf
Suppressing unfixable races
If the bug cannot be fixed, then a runtime suppression needs to be added in mozglue/build/TsanOptions.cpp. The suppressions match on the full stack, so it should be picked such that it is unique to this particular race. The bug number of this bug should also be included so we have some documentation on why this suppression was added.
Comment 1•4 years ago
|
||
Updated•4 years ago
|
Comment 2•4 years ago
|
||
The backtrace is really old so I'm not totally sure what value tsan is complaining about, but mBuiltSynthetics seems like a reasonable candidate. Really none of this seems to be aware of multithreading, so it seems like a problem that the main thread and some other thread are concurrently accessing it? (There's some allusion to multithreaded safety via multiple file descriptors, but I'm not sure that's relevant).
Permalinks to likely problematic lines:
vs
Comment 3•4 years ago
|
||
Here is a much newer log.
Comment 4•4 years ago
•
|
||
Guessing from
#0 nsZipArchive::BuildSynthetics() src/modules/libjar/nsZipArchive.cpp:810:22 (libxul.so+0x1a4b70c)
#1 nsZipArchive::FindInit(char const*, nsZipFind**) src/modules/libjar/nsZipArchive.cpp:562:17 (libxul.so+0x1a3a6c1)
#2 nsHyphenationManager::LoadPatternListFromOmnijar(mozilla::Omnijar::Type) src/intl/hyphenation/glue/nsHyphenationManager.cpp:237:8 (libxul.so+0xc354de)
#3 ...
and
#0 nsZipArchive::GetItem(char const*) src/modules/libjar/nsZipArchive.cpp (libxul.so+0x1a39d09)
#1 nsJAR::GetInputStreamWithSpec(nsTSubstring<char> const&, nsTSubstring<char> const&, nsIInputStream**) src/modules/libjar/nsJAR.cpp:269:18 (libxul.so+0x1a3a8bb)
#2
and code reading that we are accessing the same zip file, but on the main thread without going through nsJar
(which seems to add a protection for multithread usage that nsZipArchive on its own does not have).
Comment 6•4 years ago
|
||
It looks to me like it would have been a pre-existing issue; bug 1667192 touched code around there, but we were already reading from the omnijar archive prior to that.
Presumably the code in nsHyphenationManager should be migrated from using nsZipArchive to nsJar? (I haven't looked at that API to see quite what it involves...)
Comment 7•4 years ago
|
||
Bugmon Analysis:
Verified bug as reproducible on mozilla-central 20210224215151-69be3221f49a.
Failed to bisect testcase (Unable to launch the start build!):
Start: 7a5cb26a2d518e9cfaf512ba9a06239b573d7f0e (20200227033937)
End: 0e4bc84faa30e541656e8cd836965512244bd0b0 (20200730093956)
BuildFlags: BuildFlags(asan=False, tsan=True, debug=False, fuzzing=False, coverage=False, valgrind=False)
Assignee | ||
Comment 8•4 years ago
|
||
I think the correct way to move forward is to make nsZipArchive
thread-safe. Migrating to use nsJAR
is not able to fix the problem here, since nsJAR::mLock
is not used to protect the data members in nsZipArchive
.
If I am not mistaken, the only place where nsZipArchive
is touched off main thread is triggered from here. I think adding a lock at the places below should be enough.
nsZipArchive::GetItem
nsZipArchive::FindInit
Assignee | ||
Comment 9•4 years ago
|
||
Updated•4 years ago
|
Comment 10•4 years ago
|
||
Comment 11•4 years ago
|
||
bugherder |
Updated•4 years ago
|
Comment 12•3 years ago
|
||
:kershaw, since this bug contains a bisection range, could you fill (if possible) the regressed_by field?
For more information, please visit auto_nag documentation.
Comment 13•3 years ago
|
||
Sorry, wrong needinfo because of a bug in the bot.
Description
•